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SYSTEM AND METHOD FOR COMPOSING AN ELECTRONIC DOCUMENT 

FROM PHYSICAL DOCUMENTS 

Field of the invention 



The present invention generally relates to "media adaptive content", 
"electronic publishing", "print-on-demand", and to document composition and 
annotation. More particularly, the present invention discloses a system and method 
for manually designating parts of physical documents (i.e., printed document), and 
for retrieving through a network, the digital content corresponding to the designated 
parts. 



1 0 Background of the invention 



The widespread use of Internet and mobile communications offers new 
opportunities to combine electronic and printed media and to create "media 
adaptive 11 products. The philosophy behind the concept of "media adaptive" product 
is that information must be transferred to users in a form adapted to their needs. 
15 Printed documents and multimedia products are complementary and they must be 
combined depending on the users needs. To facilitate this evolution, the "electronic 
content should be accessible directly from the printed medium". 



During the last years, due mainly to the widespread use of personal 
computers and CD-ROMs and to the universal access of millions of users to the 
20 World Wide Web, the "electronic publishing" has veritably exploded. An enormous 
amount of information, including documents, books and all kinds of publications are 
now accessible to users of personal computers or specialized e-book readers. 



Even if the enthusiasm of the public for computer-based reading has been 
considered by many analysts as a threat to conventional forms of hard-copied 
25 publishing, for most people, reading paper cannot be compared with reading an 
electronic media (e.g., an e-book by using a MS Reader). Paper is portable, 
available everywhere, can be easily used and manipulated and provides a high 
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resolution. Most people prefer to read paper, whether they are familiar with 
computers or not. 

Notwithstanding the advances made in computer technology in the field of 
recording and processing data, the use of paper has not decreased. In fact, people 
5 are used to browse through paper catalogs, magazines, newspapers, maps and 
books, to flip through the pages and to glance at pictures and text. For example, a 
collection of printed color photographs can be much easily and quickly browsed than 
a sequence of computer screens. Therefore, paper continues to be the means the 
most widely used to display information. Moreover, because a paper document is 
10 more usable than a computer screen for displaying information, it is obvious that a 
large portion of the paper accumulation results from the print of electronic 
documents. Most people perform their daily work (such as meetings, projects, 
presentations, annotations, corrections) on paper and use printed copies instead of 
original electronic documents. 

15 In the last years, a new publishing technology named "Printing-on-demand", 

has contributed to keep alive this preference for printed documents. 
Print-on-demand is in fact a new printing technology, and also a new form of 
information distribution that makes possible the production of very low print runs. It is 
a production system where just the required number of copies is printed. Unlike 

20 so-called "short-run" printing (traditional book printing in small batches (50 - 100)) 
for very limited stockholding, print-on-demand enables the printing of a single book 
in response to a particular customer order. That means that on-demand titles never 
goes into stock. The ultimate step in print-on-demand, is the production of one copy 
at a time. This technology dramatically reduces the storage of hard copies and thus 

25 reduces the inventories, the distribution costs and the expenses for recycling or 
destroying unused copies. Moreover this technology enables on-demand reprinting, 
which means that titles no longer fall "out of print" . 

As a conclusion, even if many electronic document systems (e.g., Web 
browsers, e-books readers) attempt to replace paper by providing new applications 
30 such as having access to multimedia information and services, a lot of users still 
work with paper. Many analysts admit that: "consumers will insist on printing most of 
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their documents" (e.g., e-books, for reading, for browsing, for consulting, for 
annotating). 

Now, when an electronic document is printed, the advantages related to the 
use of an electronic document disappear. In particular, it is no more possible to edit, 
5 select and mark parts of the document, and to copy the selected parts for 
composing another electronic document. For example, when an electronic document 
is printed, it becomes impossible for the user to select parts of the printed copy with 
text or figures and to retrieve the digital information corresponding to the selected 
parts and to copy this information into another electronic document. 

10 In general terms, there is a real need to provide users with new systems and 

methods for improving physical copies of electronic documents, (e.g., e-books, Web 
pages, Lotus Notes, MS Word, Lotus WordPro or Adobe Portable Document Format 
(PDF)). More particularly, there is a real need to provide users with the capability of 
exporting selected pieces of information printed on physical documents (e.g., 

15 figures, tables, references, paragraphs) into electronic documents. In particular, 
there is a need to select from a physical document, information such as text and 
figures and to retrieve this information from the electronic copy of this physical 
document. 

For example, a user seating on a public library, may wish to select portions of 
20 different printed publications (e.g., some relevant paragraphs, references, and 
illustrations printed on "Celtic Legends" related books). He may wish to store data 
identifying said selections on a stand-alone (i.e., not connected to a network) 
handheld device (e.g., on a handheld computer, or a personal digital assistant). The 
same user may also wish to download at a later time, from his stand-alone handheld 
25 device to a personal computer or workstation connected to Internet, the data 
identifying the selections he made on the physical documents and to retrieve 
through the Web and from the electronic copies of the different printed documents, 
the electronic information comprised in the selected portions. 

Therefore, there is a need for new systems and methods to enable a user to 
30 compose an electronic document from information directly selected on physical 
documents. 
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None of the methods referenced in the prior art describes a system for 
directly designating and selecting parts of physical documents, and then 
automatically retrieving the digital information related to the selected portions. 

More information is available in the following publications incorporated herewith by 
5 reference: 

"Paper Is Still With Us", by Philippa Jane Benson, The Journal of Electronic 
Publishing, University of Michigan Press, at: 
http://www.press.umich.edu/jep/07-02/benson0702.html 

10 "Electronic Journals : What Do Users Think of Them ? ", Dr Cliff McKnight, 
Department of Information and Library Studies, Loughborough University, 
Loughborough, Leics LE11 3TU, UK, at: 
http://www. dl. ulis.ac.jp/iSDL97/proceedings/mcknight.htmi 

XRCE: Technical Report: EPC-1 997-1 01 (Xerox) 
15 "A Comparison of Reading Paper and On-Line Documents", at: 
http://www.xrce. xerox, com/publis/cam-trs/html/epc- 1 997- 101.htm 

"Fourteen Users in Search of a Newspaper: the Effect of Expectation on Online 
Behaviour", Ann Light, School of Cognitive and Computing Sciences, University of 
Sussex, CSRP 507, at: http://www.cogs.susx.acMk/users/ann/yExpectationsMm 

« 

20 Objects of the invention 

It is a broad object of the present invention to improve the current systems 
and methods for editing, referencing, annotating and composing documents. 

It is an object of the present invention to compose an electronic document 

4 

from information directly selected from a plurality of physical documents. 
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It is another object of the invention to identify in electronic copies of 
documents, information previously selected from physical copies of said documents. 



It is another object of the present invention to retrieve from electronic copies 
5 of documents, the information selected from the physical copies of said documents. 

It is a further object of the present invention to use portable, non connected, 
stand-alone devices, to select parts of documents from physical copies, and to store 
means for identifying said parts in the electronic copies of said documents. 

It is a still a further object of the present invention to retrieve hyperlinked parts 
10 of hyperlinked documents (e.g., portions of HTML documents, including hyperlinks) 
from selections made on physical copies of said hyperlinked documents. 

Summary of the invention 

As defined in independent claims, the present invention relates generally to a 
method, a system and a computer program for composing an electronic document 
15 from the content of one or a plurality of parts of one or a plurality of physical 
documents selected by a user. The method comprises the step of : 

• creating, for each selection made by the user, a record in a table, said record 
comprising means for associating a part selected by the user with its electronic 
copy, said step, for use in a computer device, comprising the further steps of : 
20 for each selected document : 

• receiving means identifying the document selected by the user; 

• receiving means for accessing an electronic copy of said document; 

• receiving means identifying one or a plurality of pages selected by the user of 
a physical copy of said document, said physical document comprising one or 

25 a plurality of pages; 

for each selected page : 



5 



WO 2005/001710 PCT/EP2004/006905 

• receiving means defining on the selected page, one or a plurality of parts 
selected by the user; 

for each selected part : 

• storing said means defining the selected part in a selections table; 

5 • associating in said selections table, said means defining the selected part 
with : 

• said means identifying the selected document; 

• said means identifying the selected page; 

• said means for accessing the electronic copy of said document. 

10 The method comprises the further step of: 

• retrieving an electronic copy corresponding to a part of a physical document 
selected by the user, said step comprising the steps of: 

• receiving means designating a record in the selections table; 

• identifying in the designated record : 
15 • means identifying a document; 

• means identifying a page of the document; 

• means for accessing an electronic copy of the document; 

• retrieving the electronic copy of the identified page from the electronic copy of 
the identified document; 

20 • identifying, in the designated record, means defining a part of the identified 

page; 

• marking said part on the retrieved electronic copy of the identified page; 

• copying the marked part of the electronic copy of the page into a destination 
document. 

25 Further embodiments of the invention are provided in the appended 

dependent claims. 
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Brief description of the drawings 

The novel and inventive features believed characteristics of the invention are 
set forth in the appended claims. The invention itself, however, as well as a 
preferred mode of use, further objects and advantages thereof, will best be 
5 understood by reference to the following detailed description of an illustrative 
detailed embodiment when read in conjunction with the accompanying drawings, 
wherein: 

• Figure 1 shows a stand-alone (i.e., not connected to a network) device used to 
select parts (e.g., textual paragraphs, pictures, tables) of physical documents 

10 according to the present invention. 

• Figure 2 illustrates how a workstation connected to the network, retrieves the 
digital information corresponding to the selections previously made by the user 
on the physical documents according to the present invention. 

• Figure 3 shows how a user identifies on a handheld device, a physical 
15 document. 

• Figure 4 shows how a user identifies on a handheld user device, a page of a 
physical document. 

• Figure 5 shows how a user aligns a transparent ELDT over a selected page of a 
document. 

20 • Figure 6 shows how a user marks a part of a physical document on the ELDT 
device by means of a stylus. 

• Figure 7 illustrates how a plurality of parts of different pages in a same or 
different physical documents are selected and recorded in a Selections Table 
located on the user device according to the present invention. 
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• Figure 8 shows how data stored in the Selections Table, is transferred from the 
handheld device to a workstation connected to the Internet network. 

• Figure 9 illustrates how, for each selection recorded in the Selections Table, a 
request is sent to a Publisher Server to retrieve the electronic page containing 
the information corresponding to the selection made in the physical document. 

• Figure 10 illustrates how, using coordinates stored in the Selections Table, the 
corresponding region is marked on the electronic page, and how the digital 
content of this region is copied into the clipboard buffer of the user system 
according to the present invention. 

• Figure 11 illustrates how the user, pointing to the region marked on the received 
electronic page, copies or pastes the digital information corresponding to the 
selection made in the physical document. 

• Figure 12 shows the internal structure and the principle of operation of an 
Electro-Luminescent Digitizing Tablet (ELDT). 

• Figure 13 shows the steps of selecting a part of a physical document and of 
recording said selection in a Selections table according to the present invention. 

• Figure 14 shows the steps of retrieving the electronic copy of a part of a 
physical document selected by a user according to the present invention. 

■ 

Detailed description of the preferred embodiment 

The invention itself, as well as a preferred mode of use, objects and 
advantages thereof, can be understood by reference to the following illustrative 
embodiment described by the accompanying drawings. 

The present invention addresses the need of retrieving digital copies of 
information contained in physical documents. The digital information may be an 
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electronic representation of text and images printed on physical documents. This 
electronic representation may include text, graphical elements (such as charts, 
graphs or illustrations), hypertext ... 

The invention improves the current systems and methods for editing, 
5 referencing, annotating and composing electronic documents, by allowing users to 
compose an electronic document from information directly selected from one or a 
plurality of physical documents. 

Referring to Figures 1, 2, 6, 7 and 8, the present invention can be illustrated 
with the following practical example : a user seating in a public library, selects parts 

10 of different printed publications (101) (e.g., some relevant paragraphs (605), 
references, and illustrations (705) printed on "Celtic Legends" related books), and 
stores these selections (105) on an stand-alone device (104) (e.g., on a cellular 
phone). Then, this same user downloads, from his stand-alone device (801) to a 
workstation (802) (e.g., a personal computer) connected to the Internet network 

15 (804), the selections (803) previously made on the printed publications (101). Finally, 
he retrieves through the Web from one or a plurality of servers (206), the electronic 
information corresponding to parts (204) selected in the different printed publications 
and copies this electronic information into an electronic document. 

Apart of retrieving the digital information comprised in parts of physical 
20 documents, an advantage of the present invention is the possibility to retrieve parts 
of HTML (HyperText Markup Language) documents comprising hyperlinks. 

According to a first aspect of the invention, a system and method is disclosed 
for selecting parts (605) of physical documents (601) and for creating, for eaoh 
selection made by the user, a record in a Selections Table (604). Said record 
25 comprises means for associating the part (605) selected by the user with its 
electronic copy. The method, for use in a computer device, of selecting a new part of 
a physical document and creating a new record in the Selections table 
corresponding to this new selection, comprises the steps of: 
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• identifying a physical document selected by a user, said physical document 
comprising one or a plurality of pages. 

• identifying an electronic copy of said physical document, preferably by 
identifying : 

• a publisher server network address (i.e., the URL of the Publisher Server), 
and 

• a document number. 

In a preferred embodiment, the network address of the publisher server and the 
document number are printed on said physical document and are entered by the 
user in the computer device. 

• identifying a page selected by the user. 

In a preferred embodiment, the page number is entered by the user in the 
computer device. 

• identifying a part selected by the user on a page of said physical document. Said 
step comprises the steps of : 

• placing aligned a transparent electro-luminescent digitizing tablet (ELDT) over 
(or under) the selected page; 

• identifying the selected part of the document. 

In a preferred embodiment, the user draws over the ELDT, a polygonal line 
around the selected part by means of a stylus which senses and transmits the 
coordinates of the polygon vertices to the computer device. 

• recording in a Selections Table, preferably located on the computer device, 
means for identifying the electronic copy of the physical document and for 
retrieving the digital information corresponding to the part of the physical 
document selected by the user, said means comprising : 

• the publisher server network address, 

• the document number; 

• the number of the page selected by the user; and 

• the vertices coordinates of the polygon drawn by the user to mark the part of 
the page selected by the user in the physical document. 

According to a second aspect of the invention, a system and method is 
disclosed for retrieving the electronic copy (1008) corresponding to a part (705) of a 
physical document (701) selected by a user. For each record in the Selections Table 

10 
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(1003), the method of retrieving the electronic copy of a selection made by the user 
from the corresponding electronic document stored on the Publisher Server (1005), 
comprises the steps of: 

• retrieving the electronic page from the electronic document located on the 
5 Publisher Server using : 

• the Publisher Server network address (URL), 

• the document number, and 

• the number of the page selected by the user 
previously recorded in the Selections Table. 

10 • identifying in the received electronic page, the electronic information 
corresponding to the selected part of the physical document, using the vertices 
coordinates of the polygon drawn by the user around said selected part, said 
coordinates being recorded in the Selections Table. 

• copying the identified electronic information into a buffer, preferably the operating 
15 system clipboard. 

• pasting the buffer content into a destination determined by the user. 

Components of the system according to the present invention 

As shown in Figures 1 and 2, the system according to a particular 
embodiment of the present invention, comprises the following components: 

• one or a plurality of physical, i.e., hard-copy, printed documents (101) (e.g., 
books) comprising one or a plurality of pages. 

• a physical document interface comprising a transparent Electro-Luminescent 
Digitizing Tablet (ELDT) (102) (to be placed over a page of a selected 
physical document) for capturing and digitizing strokes made by the user to 
mark parts of physical documents. 

• a stylus (103) to make strokes on the ELDT (102) to mark parts of the 
physical documents. 

• a user device (104), connected (106) to the ELDT interface (102) for storing 
annotations and strokes made on the ELDT by the user with the stylus. 

• a user workstation (201 ). 
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• a Publisher Server (206) (e.g., a Web server). 

• a communication network (207) (e.g., the Internet network). 

The main components of the disclosed system and their operation are 
described hereafter. 

5 

Physical Documents 

Physical documents (101) can be of any kind, for example, newspapers, legal 
documents, maps (e.g., topographical maps, political maps, historical maps, route 
maps, shaded relief maps, city maps, natural resources maps, rail road maps or 

10 even any other type of map), fiction novels, academic text books, technical books, 
commercial catalogs or even any other type of engraved, written, or printed surface. 
The material of the document can be paper, plastic, wood or any other material. The 
electronic version of the physical document (301 ) is stored on a Publisher Server 
(206). To identify from a physical document (301), its corresponding electronic 

15 version : 

• the URL (i.e., the Internet address) (306) of the Publisher Server (206) where this 
electronic version, is located; and 

• a reference to the document (307) (e.g., the ISBN number, or any other name, 
number, reference) 

20 are printed on or attached to the physical document (e.g., on the front cover, back 
cover or first page). 

Transparent Electro-Luminescent Digitizing Tablet (ELDT) 

As shown in Figure 12, in a preferred embodiment the Electro-Luminescent 
Digitizing Tablet (ELDT) (1200) comprises two superposed, functionally 
25 independent, transparent foils, namely: 

• a transparent Digitizing Tablet (DT) (1201), of the type commonly used to 
manufacture Position Sensible Liquid Crystal Display Devices (PSLCDs) for 
computers. The generated signals are generally proportional to the 
coordinates of the point pressed (1204) by the stylus (1208), and 
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• a transparent electro-luminescent display (EL) (1202), which is a transparent, 
bright, self-emitting display that can emit light (1205) from either one or both 
surfaces. 

The combination of both foils (i.e., the digitizing tablet (1201) stacked over the 
5 electro-luminescent display (1202)) forms an electro-luminescent digitizing tablet 
(ELDT) (1200). Figure 12 represents an ELDT placed and aligned over a physical 
document (1203) comprising a plurality of items (1207) (i.e., words, pictures, icons, 
etc.) printed (or written, painted, engraved ...) on its surface. This figure illustrates 
how the electro-luminiscent display (1202) emits light (1205) at the points (1204) 
10 pressed by the stylus (1208),. The points pressed by the stylus are highlighted over 
the physical document (1 203), while the remainder parts of the ELDT, not being 
marked (or pressed) by the stylus, transmit light (1206) through both transparent 
foils (1201), (1202), so that the surface of the physical document is visible on non 
marked (or pressed) regions. 

15 The ELDT may communicate (106) with the user device (104) over an 

infrared link, a serial wired connection or any other communication means (e.g. by 
means of a wireless connection operating in the globally available 2.4 Ghz band of 
the "Bluetooth" specification, as promoted by the "Bluetooth Special Interest Group" 
and documented on the Official Bluetooth Website http://www.bluetooth.com/). 

« 

20 Transparent Digitizing Tablets 

Digitizing tablets are the electronic equivalents of the familiar pencil and 
paper used for drawing, tracing, or sketching. The movement of a pencil-like stylus 
over a tablet surface re-creates a drawing on the computer screen. Drawings may 
then be imported to graphics or presentation application programs and be edited, 

25 formatted, and printed. 

According to an aspect of the present invention, the transparent digitizing 
tablet (1201) may be commonly built on a substrate having a matrix formed by 

• a plurality of first lines formed over a plurality of scanning lines, and 

• a plurality of second lines formed over a plurality of data lines. 
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The first and second lines are used to determine the position of an applied stylus. 
The transparent lines and contacts are made by a transparent conductive material, 
e.g., indium tin oxide (ITO). When integrated on top of a display surface, a 
transparent digitizing tablet is actually a layer that has a mesh of transparent wire 
5 sensors running through it and that sometimes visually looks like moiree patterns on 
the top of the display. These thin wires in conjunction with the pen produce 
movement and button events that are reported to the computer. 

Transparent digitizing tablets are produced, for example, by Calcomp and 
Wacom. A transparent digitizing tablet technology it is possible to use for 
10 implementing the present invention, is the one used by the Wacom Technology 
Company. The manufacturing of pen tablet systems is described on the Web site of 
Wacom Technology Co. at http://www.wacom.com/lcdtablets/. 

Transparent Electro-Luminescent Displays 

According to another aspect of the present invention, the transparent 
15 electro-luminescent display (1202) may include a substrate having an array formed 
by 

• a plurality of transparent scanning lines, 

• transparent data lines crossing said scanning lines, and 

• electro-luminescent (EL) elements (pixels) on the intersections of the scanning 
20 and data lines. 

With today's technology, this passive-matrix, light-emitting display may be made of 
an array of TOLED's (Transparent Organic Light Emitting Devices) of the types used 
to create vision area displays on windshields, cockpits, helmets and eyeglasses. 

In its most basic form, a TOLED is a monolithic, solid-state device consisting 
25 of a series of "small molecule" organic thin films sandwiched between two 
transparent, conductive layers. When a voltage is applied across the device, it emits 
light. This light emission is based upon a luminescence phenomenon wherein 
electrons and holes are injected and migrate from the contacts toward the organic 
heterojunction under the applied electric field. When these carriers meet, they form 
30 excitons (electron-hole pairs) that recombine radiatively to emit light. As a result, 
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TOLEDs are bright, self-emitting displays that can be directed to emit from either or 
both surfaces. This is possible because, in addition to having transparent contacts, 
the organic materials are also transparent over their own emission spectrum and 
throughout most of the visible spectrum. 

5 TOLED displays are today manufactured with standard silicon 

semiconductors. Since TOLEDs are thin-film, solid-state devices, they are very thin, 
lightweight and durable, ideal for portable applications, like the one disclosed in the 
present invention. TOLEDs can be bottom, top, or both bottom and top emitting. 
Also, TOLEDs technology has attractive advantages regarding, transparency 

10 (TOLED displays can be nearly as clear as the glass or substrate they're on and 
when built between glass plates, TOLEDs are > 85% transparent when turned off), 
energy efficiency (for longer battery life), full viewing angle, bright and high contrast 
light emission, fast response time, and environmental robustness. Thus, TOLEDs 
are well suited for. manufacturing the light-emitting, electro-luminescent, display 

15 component, used jointly with the transparent digitizing tablet accordingly with the 
present of this invention. 

One example of light emitting foil technology it would be possible to use is the 
technology used for the TOLEDs manufactured by UNIVERSAL DISPLAY 
CORPORATION, fully described in the Web site of UNIVERSAL DISPLAY 
20 CORPORATION at http://www.universaldisplay.com/toled.php 

Stylus 

Pen like stylus (1208) are commonly used as input devices for data 
processing and storage systems in place of conventional keyboards and mouse 
devices. The stylus (1208) is used in combination with the digitizing tablet (1201) 

25 part of the ELDT (1200). The digitizing tablet (1201) incorporates a resistive or 
capacitive digitizer or sheet material. As such, information can be input by writing 
with the stylus on the ELDT device. The electro-luminescent (1 202) component of 
the ELDT displays the instantaneous position, and path of movement, of the stylus. 
In this way, the ELDT device displays the pattern traced thereon (e.g. a written 

30 message, sketch or signature). 
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One example of stylus it is possible to use, in combination with a transparent 
digitizing tablet, is the wireless, pressure sensitive Wacom's Pen. 

European patent application 014-80135.1, filed 21/12/2001 / US application 
10/317504, filed 1 2/1 2/2002 entitled "Device, method and system for retrieving and 
5 displaying handwritten annotations" (IBM's reference FR9 2001 0048) discloses an 
apparatus for use with a computer system, for capturing and displaying handwritten 
annotations, comprising a transparent digitizing tablet adapted to cooperate with a 
stylus. It also comprises a transparent electro-luminescent display adapted to 
display captured handwritten annotations, the transparent digitizing tablet being 

10 superposed on the transparent electro-luminescent display. Thanks to the 
transparent electro-luminescent display and the transparent digitizing tablet 
superposed thereon, handwritten annotations can be marked and displayed on a 
physical document without physically marking the document. Thus, a user can make 
annotations over a physical document using "electronic ink" so that what is being 

15 "written" by the user on the physical document (on the digitizing tablet) can appear 
to the user electronically illuminated by the transparent electro-luminescent display, 
on the same physical document. Such an apparatus enables the user to draw, 
sketch, annotate, construct, outline, enhance erase, associate and display 
information directly over physical documents while keeping intact the integrity of said 

20 physical documents. 

User Device 

The user device (104) can be a personal computer, an Internet enabled cell 
phone, a PDA, a network computer, an Internet appliance or a wireless IP enabled 
device. The user device can be used alone or can be connected to a network (e.g. 
25 to the Internet network). However, the user device must include means (106) for 
connecting the ELDT device (1 02) and for receiving and storing the selections made 
by the user on the physical documents (101). 

It is important to note that the user device (104) can be a simple stand-alone 
handheld device, (such as a PDA, or a cell phone) with a small memory, a small 
30 display (even lacking a graphics display), just for handling : 
• the URLs (306) of the Publisher Servers (206), 
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• the document numbers (307), 

• the selected page numbers (406) , and 

• the coordinates of the points (A, B, C, D) defining the polygon (605), (705) drawn 
by the user around the selected parts (by means of the interface provided by the 

5 combination of the ELDT (1 02) and stylus(1 03)). 

In such embodiment, the user device (104) must include means for transferring the 
recorded selections data to a user workstation (201) connected to a network (e.g. to 
the Internet network). 

User Workstation 

10 The user workstation (802) can be a personal computer connected to a 

network (804) (e.g. to the Internet network). The user workstation includes means 
for receiving information stored on the user device (801). 

Publisher Server 

The Publisher Server (206) is primarily the repository where electronic copies 
15 of physical documents (101) are stored. The electronic copy of a physical document 
includes an exact replica of the original document. The electronic representation 
may also include text, graphical elements (such as charts, graphs or illustrations), 
hypertext (i.e., hyperlinked objects) ... 

Each physical copy of a document (or publication) is associated with an 
20 electronic copy of this document by means of : 

• the network address (i.e., the URL or Internet address) (306) of the Publisher 
Server (206), and 

• a reference to the document (307) (e.g., the ISBN number, or any other), printed 
or attached to physical document (301) (e.g., on the front cover, back cover or 

25 first page). 

Method of selecting parts of physical documents and of recording said 
selections in a Selections Table 
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As shown in Figure 13, the present invention discloses a system, method 

■ 

and computer program for manually selecting and associating parts of physical 
documents with electronic copies of said printed parts. 

The method, for use in a computer system, comprises the steps of: 

5 • (1301) identifying a physical document selected by a user. In a preferred 
embodiment, a document identifier is printed on the physical document and 
entered in the computer system by the user. 
« storing the document identifier in a Selections Table. 

• (1302) identifying a network address where an electronic copy of said 

10 physical document can be accessed. In a preferred embodiment this network 

address is printed on the physical document and entered in the computer 
system by the user. 

• (1 302) storing in the Selections Table, said network address; 

• (1303) identifying a page selected by the user in said physical document, and 
15 storing means for identifying said page in the Selections Table. 

• (1 304) identifying the part of said page selected by the user : 

• (1 305) the transparent ELDT is placed aligned over the selected page by 
the user so that the user can see the page through the transparent ELDT 
device. 

20 • (1306) The user, using the stylus, draws over the transparent ELDT, a 

polygonal line around the selected part. 

• (1307) the coordinates of the selected part (or region) are captured (i.e., 
digitized) and displayed by means of the ELDT. 

• (1308) receiving from the ELDT the coordinates of the selected part (or 
25 region), and 

• storing the coordinates associated with the selected page and document, in 
the Selections Table. 

Figure 1 shows the main components of the invention for creating means for 
referencing electronic information related to selected parts of physical documents 
30 (101). In a preferred embodiment, the main components are not connected to the 
Internet network and basically comprise : 
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• a transparent Electro-Luminescent Digitizing Tablet (ELDT) (1 02), 

• a stylus (103), and 

• a handheld user device (104) (e.g., a PDA, a hand-held computer, a cell phone). 

Figure 3 shows how the user selects a physical document (or publication) 
5 (301) (e.g., printed book entitled "Celtic Legends"). By means of any user interface 
(keyboard, mouse, voice recognition software and microphone,...) or any other 
reading means (e.g., barcode reader ...), the user enters a code or name for 
identifying the document. In the preferred embodiment shown in Figure 3, the user 
opens a new record in a table called "Selections Table" (305) located on the user 
10 device (304) and types : 

• the publication title (e.g., "Celtic Legends") (308), 

• the address on the Web (e.g., the URL "http.//www.virgin.nef) (306) of the 
Publisher Server where the electronic copy of the physical document (301) is 
stored, and 

15 • the reference number (or ISBN) (e.g., "170301" ) (307) of the electronic 
document stored in the Publisher Server. This electronic document is a copy 
(with the same format) of the selected physical document (301). 
The URL (306) of the Publisher Server and the document reference number (307), 
used to identify the document and locate the electronic document copy through the 
20 Web, must be printed or attached at a predefined reserved place on the physical 
document (301) (e.g., on the front cover, back cover or first page). For each new 
selection made by the user in a given physical document (301), a new entry must be 
created in the Selections Table (305). 

Figure 4 shows how the user while browsing or reading a physical document 
25 (401), finds on a page, an interesting paragraph (406) and transcribes the content of 
this paragraph to another document or application. To create a reference to this 
interesting paragraph, the user first identifies the selected page by entering the Page 
Number (e.g., page "16") (406) on the user device (404). The Page Number (406) is 
then recorded in the Selections Table (405). 

30 As shown in Figure 5, after entering on the user device (504), the number of 

the selected page, the ELDT (502) is then: 
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• placed by the user over the selected page, and 

• aligned (506) with the borders of this page by some conventional means (e.g., 
by adjusting the upper left corner of the ELDT device with the upper left 
corner of the page). 

5 After this operation, the user can see the content of the selected page 

through the transparent ELDT. 

Figure 6 shows how to identify the part (406) of the physical document (601) 
selected by the user. Keeping the transparent ELDT device (602) aligned over the 
selected page, the user marks on this page, by means of the stylus, the selected 

10 part (605) (e.g., the user marks the points "A", "B", "C", "D", of the polygonal 
region drawn around the selected region) . The coordinates of the points (e.g., "A", 
"B", "C", "D") defining the polygonal region are sensed by the ELDT device (602) 
and are transmitted to the user device (603). The coordinates are recorded in the 
Selections Table (604), and are associated with the selected document and page 

15 identification means. 

Figure 7 shows how, using the same procedure, different parts of different 
pages of a same or different physical documents can be selected by the user. For 
each selection made by the user on a physical document (701), 

• the publication title (e.g. , "Celtic Legends 99 ) (308), 

20 • the network address (URL) of the Publisher Server {e.g./'httpJ/www.virgin.nef) 
(306), 

• the document number (e.g., "170301") (307), 

• the page number (e.g., page "77'% and 

• the coordinates of the vertices defining the polygonal region around the selected 
25 part of physical document (e.g., "A", "B", "C", "D"), 

are recorded on a new record on the Selections Table (704). 

Method of retrieving electronic copies corresponding to parts of electronic 
documents selected by the user 
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As shown in Figure 14, the present invention discloses also a system, 
method and computer program for retrieving through a network (e.g., through 
Internet) the digital content of parts of physical documents selected by a user. 

The method comprises the steps of: 

5 • (1401) transmitting the Selections Table from the user device to a network 
connected user workstation; 

for retrieving the electronic copy of a selection recorded in the Selections Table: 

• (1402) sending a request to the Publisher Server to receive from the electronic 
document, the selected electronic page containing the digital content of the 

10 selection; 

• (1403) receiving from the Publisher Server the requested electronic page, 

• (1 403) displaying said electronic page on the user workstation; 

• (1404) marking the selected region on the received electronic page; the 
coordinates of the region selected on the physical document being stored in the 

15 Selections Table; 

• (1405) copying into the operating system clipboard, the content of the region 
marked on the electronic page; 

• (1406) copying (or cutting) and pasting the clipboard content in the user 
designated destination. 

20 Figure 2 shows how, according to a preferred embodiment of the invention, a 

workstation (201) connected to a network (e.g., to the Internet (207)) is used for 
retrieving from the electronic documents stored on the Publisher Servers (206), the 
digital information (204) corresponding to the selections (705) previously made by 
the user on the physical documents (by means of components (102), (103), (104)). 

25 Figure 8 illustrates how, according to a preferred embodiment of present 

invention, once selections in physical documents (e.g., text paragraphs, pictures, 
tables) have been performed by means of the stand-alone components (102) 
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(103)(104), data recorded in the Selections Table (803) located on the user device 
(801), is transmitted to a workstation (802) connected to the internet network (804). 

Figure 9 illustrates how the user selects a record in the Selections Table 
(903) stored on the user workstation (901), and retrieve from the Publisher Server 
5 (905), the digital content of the selection. As illustrated by this figure, using the 
information stored in the Selections Table (903) : 

• the URL of the Publisher Server (e.g., "httpJ/www.virgin.net') (306), 

• the document number (e.g., "170301") (307), and 

• the selected page number (e.g., page "17"), 
10 The user workstation sends a request (e.g., 

"http://www.virgin.net/170301/?&Page=17" ) (904) through the Internet network 
(907) to the Publisher Server (905), to retrieve the selected electronic page (i.e., 
page "17") from the selected electronic document (i.e., "170301" ). The electronic 
document is the exact replica on the Publisher Server (905) of physical document 
15 (entitled "Celtic Legends"). As illustrated in this same figure, the requested 
electronic page (906) is retrieved from the electronic document stored on the 
Publisher Server (905) and transmitted to the user workstation (901) to be displayed. 

Figure 10 shows how when an electronic page (1006) of a document is 
retrieved from the Publisher Server (1005) and displayed on the user workstation 

20 (1001), a plug-in program resident on this user workstation (1001) marks the 
selected region (1008) on the received electronic page (1006), and then copies the 
digital content of this region into the clipboard of the operating system. It must be 
noted that plug-in programs for controlling mouse coordinates and actions (such as 
marking one or a plurality of rectangular regions on the screen using screen 

25 coordinates) can be implemented using, for instance, assembler code, or open 
-free- code from "C" utilities. The plug-in program marks the selected region using 
the coordinates (1007) of the points (e.g., "A", "B", "C", "D") defining the polygon 
enclosing the region (705) selected by the user by means of the ELDT device (702) 
on the selected page of the physical document (701), 

30 Figure 1 1 illustrates how, alternatively, the user points (e.g., by means of the 

mouse) (1105) to the region marked (1104) on the received electronic page (1103) 
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(this region corresponds to the region marked by the user (705) on the 
corresponding physical document (701)), and then "copies" it (1106) (or "cuts") and 
"pastes" it in an electronic document (1107) or any other data repository or 
application on the user workstation. 

5 It must be noted that the standard COPY (or even CUT) commands of most 

operating systems saves the selection in the clipboard. Also, the standard PASTE 
command transfers the contents of the clipboard to the current page of a document, 
either at the current insertion point or in place of the selection. If the clipboard 
comprises URLs (hyperlinks), these URLs are also included in the pasted data.. 

10 What has been described is merely illustrative of the application of the 

principles of the present invention. Other arrangements and methods can be 
implemented by those skilled in the art without departing from the spirit and scope of 
the present invention. 
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